Improving Long-Term Online Prediction with Decoupled Extended Kalman Filters
نویسندگان
چکیده
Long Short-Term Memory (LSTM) recurrent neural networks (RNNs) outperform traditional RNNs when dealing with sequences involving not only short-term but also long-term dependencies. The decoupled extended Kalman filter learning algorithm (DEKF) works well in online environments and reduces significantly the number of training steps when compared to the standard gradient-descent algorithms. Previous work on LSTM, however, has always used a form of gradient descent and has not focused on true online situations. Here we combine LSTM with DEKF and show that this new hybrid improves upon the original learning algorithm when applied to online processing.
منابع مشابه
Kalman filters improve LSTM network performance in problems unsolvable by traditional recurrent nets
The long short-term memory (LSTM) network trained by gradient descent solves difficult problems which traditional recurrent neural networks in general cannot. We have recently observed that the decoupled extended Kalman filter training algorithm allows for even better performance, reducing significantly the number of training steps when compared to the original gradient descent training algorit...
متن کاملOnline Symbolic-Sequence Prediction with Recurrent Neural Networks
This paper studies the use of recurrent neural networks for predicting the next symbol in a sequence. The focus is on online prediction, a task much harder than the classical offline grammatical inference with neural networks. Different kinds of sequence sources are considered: finitestate machines, chaotic sources, and texts in human language. Two algorithms are used for network training: real...
متن کاملOnline Symbolic-Sequence Prediction with Discrete-Time Recurrent Neural Networks
This paper studies the use of discrete-time recurrent neural networks for predicting the next symbol in a sequence. The focus is on online prediction, a task much harder than the classical offline grammatical inference with neural networks. The results obtained show that the performance of recurrent networks working online is acceptable when sequences come from finite-state machines or even fro...
متن کاملCombing Extended Kalman Filters and Support Vector Machines for Online Option Price Forecasting
This study combines extended Kalman filters (EKFs) and support vector machines (SVMs) to implement a fast online predictor for option prices. The EKF is used to infer latent variables and makes a prediction based on the Black-Scholes formula, while the SVM is employed to capture the nonlinear residuals between the actual option prices and the EKF predictions. Taking option data traded in Taiwan...
متن کامل